This application is related generally to the development of software. More specifically, this application is related to methods and systems for testing software development.
One of the difficulties in developing software applications, particularly large software applications, is the potential for causing unintended errors with parts of the code as changes or additions are made to the code during development. Modern software codes can frequently include millions of lines of code and can be intended to process data-set collections that include hundreds of millions of elements. During development of the software, potentially dozens or more software developers may work on portions of the code, often in isolation from each other. It is a significant challenge during such initial development to ensure that the resulting code operates as desired on all of the data-set collections.
Even after the initial development of software, there is, moreover, frequently additional development that is to take place on the software. For example, the functionality of the software may be improved in response to requests from its users, or new functionality may be added to exploit the capabilities provided by the software further. This phase of development may be even more sensitive to changes that may be made in the software since it may be possible for those changes to disrupt the previously proper functioning of the software.
Significant time on the part of programmers is often spent during these development phases to test software in a variety of different circumstances to identify “bugs” and to undertake additional programming to eliminate such bugs. This testing is typically done on an ad hoc basis, with testers trying to ensure that as much relevant code as possible is tested by using test input that is as broad as possible. Because such attempts rely on the ingenuity of humans to identify the diversity of applicable circumstances, they are virtually always incomplete and, in practice, many relevant portions of the code remain untested. An alternative to relying on the ingenuity of human testers is to test the code as it is developed with the entire collection of input data sets. While this approach has the virtue of testing relevant portions of the code, it is highly inefficient. Testing with many of the data sets is redundant, and the sheer requirement of testing with a very large number of data sets is simply prohibitive because of the time it takes to do so, particularly if testing is to be performed as an ongoing part of the development project.
There is, accordingly, a general need in the art for methods and systems that permit more efficient testing during software development.